{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# DAC Contest\n",
    "This reference design will help you walk through a design flow of DAC SDC 2020.\n",
    "\n",
    "This is a simplified design to help users get started on the FPGA platform and to understand the overall flow. It does not contain any object detection hardware.\n",
    "\n",
    "If you have any questions, please post on the [Piazza page](piazza.com/dac_2018/winter2020/dacsdc2020/home).\n",
    "\n",
    "## Hardware\n",
    "The base hardware design contains the Zynq MPSoC processor (A53) with a DMA and FIFO implemented in the PL.  \n",
    "Note that in this starting hardware, no actual image processing/detection is done.\n",
    "Pictures are:\n",
    "1. sent from PS to DMA, then DMA to FIFO\n",
    "2. sent back from FIFO to DMA, then DMA to PS.\n",
    "\n",
    "Note that the FIFO should be replaced with meaningful hardware to process the pictures.\n",
    "\n",
    "You can create a Vivado project by opening Vivado 2019.1, changing to the `hw` directory, and running `source dac_sdc.tcl`.  This will create and open a project that you can edit.  Each time you submit your files, you should run `File->Project->Write Tcl...` to generate a new tcl file to submit.  Be sure to check the `Recreate Block Designs using Tcl` box, and include any custom IP modules you use.\n",
    "\n",
    "## Software\n",
    "Note:\n",
    "1. Any change in `dac_sdc.py` will make your design fail in evaluation. This file should not be changed.\n",
    "2. You can use both PS and PL side to do inference.\n",
    "\n",
    "Timing **should include the entire execution of your design, including reading the image from the SD card**, as shown in the notebook below.\n",
    "\n",
    "Please check the reference for details. You can exclude \"save_results_xml\" when counting time.\n",
    "\n",
    "Batch size is 500 by default.\n",
    "\n",
    "Please write your results using the methods `write()` and `save_results_xml()`.\n",
    "\n",
    "It is your choice how to record the inference result. \n",
    "However, it must be readable, and you must convert it to XML files. An example is provided.\n",
    "Please place your code into `teamname.ipynb`. \n",
    "Your design should be able to process all evaluation images in five cells.\n",
    "\n",
    "\n",
    "Your notebook should contain 4 code cells:\n",
    "  1. Importing all libraries and creating your Team object.\n",
    "  2. Downloading the overlay, and performany any one-time configuration.\n",
    "  3. Processing all images.\n",
    "  4. Writing results to XML and any other cleanup.\n",
    "\n",
    "The following cells are not a real design for image detection. But you can use them to start your design. (Section 3 is broken into a few cells for demonstration purposes)\n",
    "\n",
    "### 1. Importing all libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "import os\n",
    "\n",
    "sys.path.append(os.path.abspath(\"../common\"))\n",
    "\n",
    "import math\n",
    "import time\n",
    "import numpy as np\n",
    "from PIL import Image\n",
    "from matplotlib import pyplot\n",
    "import cv2\n",
    "from datetime import datetime\n",
    "\n",
    "import pynq\n",
    "import dac_sdc\n",
    "from IPython.display import display\n",
    "\n",
    "team_name = 'sample_team'\n",
    "team = dac_sdc.Team(team_name, batch_size = 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Your team directory where you can access your bitstream, notebook, and any other files you submit, is available as `team.team_dir`.**\n",
    "\n",
    "\n",
    "### 2. Preparing the overlay and weight loading\n",
    "Overlay loading must be executed in this cell.\n",
    "\n",
    "In this sample hardware, the DMA instance is exposed as an attribute of the overlay object.  You aren't required to use DMA for your hardware solution.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "overlay = pynq.Overlay(team.get_bitstream_path())\n",
    "dma = overlay.axi_dma_0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Data flows from PS to PL, then back to PS by DMA. Using an interrupt is recommended.\n",
    "\n",
    "### 3. Processing Images\n",
    "\n",
    "\n",
    "In this section we first demonstrate how to use then `opencv` library to read and manipuate images, then we show how to send an image to the hardware using DMA.  Finally, we demonstrate how to loop through the images and processing them, recording run-time and energy usage.  The final example will is most closely related to what you need to do.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "interval_time = 0\n",
    "\n",
    "# Skip image 0\n",
    "team.get_next_batch()\n",
    "\n",
    "# Get image 1\n",
    "image_path = team.get_next_batch()[0]\n",
    "\n",
    "original_image = Image.open(image_path)\n",
    "original_array = np.array(original_image)\n",
    "original_image.close()\n",
    "\n",
    "original_result = Image.fromarray(original_array)\n",
    "display(original_result)\n",
    "\n",
    "old_width, old_height = original_image.size\n",
    "print(\"Original image size: {}x{} pixels.\".format(old_height, old_width))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will now resize the image using the openCV library."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "new_width, new_height = int(old_width/2), int(old_height/2)\n",
    "original_image = Image.open(image_path)\n",
    "resized_image = original_image.resize((new_width, new_height), \n",
    "                                      Image.ANTIALIAS)\n",
    "resized_array = np.array(resized_image)\n",
    "original_image.close()\n",
    "\n",
    "resize_result = Image.fromarray(resized_array)\n",
    "display(resize_result)\n",
    "\n",
    "width, height = resized_image.size\n",
    "print(\"Resized image size: {}x{} pixels.\".format(height, width))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Pushing the picture through the pipeline\n",
    "In the following example, we will also use contiguous memory arrays for sending\n",
    "and receiving data.\n",
    "\n",
    "The size of the buffer depends on the size of the input or output data.\n",
    "Since the image we process in the following example (`0.jpg`) has 453x674 RGB pixels,\n",
    "we will use `cma_array` of the corresponding size as well."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "xlnk = pynq.Xlnk()\n",
    "in_buffer = xlnk.cma_array(shape=(453, 674, 3), dtype=np.uint8, cacheable = 1)\n",
    "out_buffer = xlnk.cma_array(shape=(453, 674, 3), dtype=np.uint8, cacheable = 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note: In the following example, we are only dealing with one image. \n",
    "We will just send one image to the FIFO and loop it back. \n",
    "In the real contest, you should process all the RGB images in every batch.\n",
    "\n",
    "Note that the `rgb_array` has to be copied into the contiguous memory array\n",
    "(deep copy)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "interval_time = 0\n",
    "\n",
    "# Get image 0\n",
    "team.reset_batch_count()\n",
    "image_path = team.get_next_batch()[0]\n",
    "\n",
    "bgr_array = cv2.imread(str(image_path))\n",
    "rgb_array = cv2.cvtColor(bgr_array, cv2.COLOR_BGR2RGB)\n",
    "in_buffer[:] = rgb_array\n",
    "\n",
    "pyplot.imshow(in_buffer)\n",
    "pyplot.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we will push the data from input buffer through the pipeline to the output buffer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def dma_transfer():\n",
    "    dma.sendchannel.transfer(in_buffer)\n",
    "    dma.recvchannel.transfer(out_buffer)    \n",
    "    dma.sendchannel.wait()\n",
    "    dma.recvchannel.wait()\n",
    "    \n",
    "\n",
    "dma_transfer()\n",
    "\n",
    "pyplot.imshow(out_buffer)\n",
    "pyplot.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Iterating through all images by batch, recording timing and energy measurements\n",
    "\n",
    "In this cell, you will:\n",
    "  * Perform any startup configuraton of your hardware (Download weights, etc.)\n",
    "  * Measure the time and energy for processing all the images. \n",
    "  \n",
    "The timer and power measurements should be running for your entire execution.  The code below shows how you can measure the total energy usage.  Make sure you follow this approach.\n",
    "\n",
    "In the following example we will time the processing and measure energy usage of a single picture."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "interval_time = 0\n",
    "total_time = 0\n",
    "total_energy = 0\n",
    "result = list()\n",
    "team.reset_batch_count()\n",
    "rails = pynq.get_rails()\n",
    "\n",
    "# Start timer and energy recorder.  If this fails, then you may not have configured\n",
    "# your system to measure power usage.  Follow the steps provided at\n",
    "# https://github.com/jgoeders/dac_sdc_2020/blob/master/support/measure_power/README.md\n",
    "start = time.time()    \n",
    "recorder = pynq.DataRecorder(rails[\"5V\"].power)\n",
    "with recorder.record(0.05):        \n",
    "    while True:\n",
    "        # get a batch of images\n",
    "        image_paths = team.get_next_batch()\n",
    "        if image_paths is None:\n",
    "            break\n",
    "\n",
    "        # Read all images in this batch from the SD card.\n",
    "        # This part doesn't count toward your time/energy usage.\n",
    "        rgb_imgs = []\n",
    "        for image_path in image_paths:\n",
    "            bgr_img = cv2.imread(str(image_path))    \n",
    "            rgb_img = cv2.cvtColor(bgr_img, cv2.COLOR_BGR2RGB)\n",
    "            rgb_imgs.append(rgb_img)\n",
    "   \n",
    "        for img in rgb_imgs:\n",
    "            in_buffer[:] = img    \n",
    "            dma_transfer()\n",
    "            \n",
    "# timer stop after batch processing is complete\n",
    "end = time.time()\n",
    "t = end - start\n",
    "    \n",
    "# Energy measurements    \n",
    "energy = recorder.frame[\"5V_power\"].mean() * t    \n",
    "\n",
    "total_time = t\n",
    "total_energy = energy\n",
    "print(\"Total time:\", total_time, \"seconds\")\n",
    "print(\"Total energy:\", total_energy, \"J\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. Recording results, cleanup\n",
    "\n",
    "It is your responsibility to record your results.  You should call `team.save_results_xml(result_list, total_runtime, total_energy)` to save your results.\n",
    "  * `results_list`: Each element in this list is a 4-integer list [xmin, xmax, ymin, ymax] indicating the bounding box location deterined by your object detection.  This list should have the same number of elements as there are images.  The first element in the list will correspond to image 0.jpg, etc.\n",
    "  * `total_runtime`: The total runtime of your processing of all images, as described above.\n",
    "  * `total_energy`: The total energy of your processing of all images, as described above.\n",
    "  \n",
    "The box below shows an example:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# These are fake object locations since the example design doesn't actually perform object detection\n",
    "result_rectangle =  [[0,358,0,263],[0,1350,0,707]]\n",
    "\n",
    "team.save_results_xml(result_rectangle, total_time, total_energy)\n",
    "print(\"XML results written successfully.\")\n",
    "\n",
    "# Remember to free the contiguous memory after usage.\n",
    "xlnk.xlnk_reset()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### What to submit\n",
    "\n",
    "Submit your:\n",
    "  * Notebook\n",
    "  * .bit and .hwh files\n",
    "  * The tcl file that can be used to create your Vivado project in 2019.1\n",
    "  * Any supporting hardware files that Vivado needs (eg. IP)\n",
    "  * Any other files your notebook needs to run (eg. weights file)\n",
    " "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
